This invention relates to a technology for generating a color image.
In recent years, research and development have been actively carried out in order to photograph a nighttime scene with an on-vehicle camera, a surveillance camera, or the like and make use thereof for safety and security of people's lives. Such a nighttime scene exhibits an extremely large contrast, and exceeds a limit of a dynamic range of a general image pickup element, which may generally cause a phenomenon called “blown-out highlights”, “blocked-up shadows”, or “color saturation”. In particular, in a case where a color (chromaticity) of a subject is used for image recognition processing or the like after photographing, when only a component of a specific color among color components such as red (R), green (G), and blue (B) is saturated, the color of a resultant image differs from an actual color of the subject, which may inhibit the color image from being formed appropriately. For example, when a red lamp existing within the subject is increased in luminance, the red component within the image is saturated. Then, the green component and the blue component increase relatively within the image, and hence a part corresponding to the red lamp appears whitish within the generated image.
In contrast, in International Publication No. WO 2010/116922 A1, there is disclosed a technology intended to provide an image input device capable of obtaining information necessary to create an appropriate color image even when the component of a specific color is saturated. In this technology, an image Img2 obtained by photographing with a first shutter speed and an image Img1 obtained by photographing with a second shutter speed higher than the first shutter speed are obtained. In those images, coordinates are compared between a saturated pixel within the image Img2 and a high luminance pixel within the image Img1. When those pixels are in the same coordinate position, it is estimated that the same subject has been photographed, and a chromaticity included in a signal from the saturated pixel having the same coordinates within the image Img2 is adjusted by using the following expressions so as to match with a chromaticity included in a signal from the high luminance pixel within the image Img1.
In other words, when a luminance signal of the high luminance pixel of a color image Img1 is set as Y1(x,y), color difference signals are set as Cb1(x,y) and Cr1(x,y), and a luminance signal output from the saturated pixel of a color image Img2 is set as Y2(x,y), corrected color difference signals Cb2′(x,y) and Cr2′(x,y) are calculated by using the following expressions.Cb2′(x,y)=Cb1(x,y)×Y2(x,y)/Y1(x,y)  (1)Cr2′(x,y)=Cr1(x,y)×Y2(x,y)/Y1(x,y)  (2)
In other words, WO 2010/116922 A1 discloses a technology for replacing the color difference signal within the color image signal output from the saturated pixel within the color image Img1 with the above-mentioned corrected color difference signal. Here, (x,y) represents coordinates on an image pickup plane of an image pickup element.
Further, in Japanese Patent Application Laid-open No. 2003-319404 A, there is disclosed a technology for combining a plurality of images in order to enhance the dynamic range of an image pickup apparatus using an image pickup element such as a CCD. This technology discloses an image pickup apparatus including: indexing means for processing each of image signals of a plurality of original images obtained by applying different exposure amounts and calculating an index corresponding to the chromaticity for each divided region formed of at least one pixel; selecting means for selecting one image signal from among the image signals of a plurality of original signals for each divided region based on the index; and image means for generating one composite image signal from the selected image signals.
Japanese Patent Application Laid-open No. 2003-319404 A also discloses that, at this time, an index C used for selecting one image signal from among the image signals of the plurality of original signals for each divided region is calculated from any one of Expressions (3) to (5) by using a luminance signal Y, color difference signals R−Y and B−Y, a luminance signal Y′ of another image to be compared, and a predetermined value α.C={(R−Y)2+(B−Y)2}/(Y+α)2  (3)C={(R−Y)2+(B−Y)2}×(Y′+α)2  (4)C={|R−Y|+|B−Y|}/(Y′+α)2  (5)